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Abstract 

In this paper we investigate networks whose evolution is governed by 
the interaction of a random assembly process and an optimization pro- 
cess. In the first process, new nodes are added one at a time and form 
connections to randomly selected old nodes. In between node additions, 
the network is rewired to minimize its pathlength. For timescales, at 
which neither the assembly nor the optimization processes are dominant, 
we find a rich variety of complex networks with power law tails in the 
degree distributions. These networks also exhibit non-trivial clustering, a 
hierarchical organization and interesting degree mixing patterns. 
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1 Introduction 

Inspired by the finding that the topologies of a surprisingly large class of complex 
systems can be described by generic classes of networks, the last decade has seen 
a very rapid development in network modelling, cf. [3] for reviews. A particular 
emphasis has been put on two ubiquitous classes of networks: scale-free (SF) 
[3] and small world (SW) [H] networks. Many real-world networks display both 
features. SF networks are characterized by a degree distribution that has a 
power law tail of the form P{k) k^" with exponents a in the range between 

2 and 3. SW networks combine short average pathlengths with a strong local 
cohesiveness expressed by high densities of triangles. 

So far, most efforts to explain the formation SF networks have concentrated 
on assembly models using variants of preferential attachement. In these it is 
usually assumed that the network is assembled node by node over time, new 
nodes establishing connections to old nodes with attachment probabilities in 
proportion to the old nodes' degrees. The seminal studies which pioneered this 
approach are [ij [T3 l [TT l [5] . 

Another line of thought to explain empirically observed network characteris- 
tics is via optimization models, assuming that the system under observation rep- 
resents the end point of some optimization process. Examples to this approach 
include various models of link cost-pathlength optimization [T3J [Tni HI IH] , net- 
work models for optimal traffic llj, linearly stable dynamics [211 E], and optimal 
synchronization |12|[6j. All of these optimization models, however, consider sys- 
tems that are static in size [Tj. For specific parameters SF networks have been 
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obtained from an optimization model which minimized a trade-off between cost 
of hnks and average pathlengths [TJ. However, SF networks from optimization 
principles only seem to arise at transition points between different parameter 
regimes. Thus, to find SF networks from static optimization usually requires 
the fine tuning of some parameter - clearly not a robust explanation. 

In this paper we consider networks where both processes: assembly and op- 
timization play a role. Naturally, the ratio of time scales of network assembly 
and optimization will be important to understand the evolution of such sys- 
tems. Clearly, if assembly is slow in comparison to the optimization process, 
the topology of the evolved network can essentially be understood by a static 
optimization problem. Contrariwise, if network assembly is fast, the assembly 
mechanism will be the dominant process which shapes network topology. As we 
discussed above both of these limiting cases have been considered extensively 
in the literature before, in which it was always assumed that time scales for 
assembly and optimization are well separated and thus, that possible interac- 
tions between the two processes can be neglected. What, however, about the 
continuum of timescales in between, where both processes interact? This is the 
problem we will address in the present study. 
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2 Model Description 

As an example for an optimization process to illustrate our point we consider 
the optimization for shortest average shortest pathlength, 



where d{i,j) is the length of the shortest path that connects the nodes i and j 
on the network. As an assembly process we consider the addition of nodes which 
form links to randomly selected nodes from the old network. For simplicity, we 
restrict the study to undirected binary links. The combination of an assembly 
and optimization mechanism appears of interest, since pathlength optimization 
typically generates star networks with one dominant hub node |141 [7] , while ran- 
dom node assembly generates networks with exponential degree distributions [4] , 
i.e. networks without hub nodes. Thus two opposing tendencies are combined: 
a pressure towards hub formation resulting from the optimization process and a 
tendency against link accumulation, stemming from the node assembly. Below 
we will explore typical networks that are formed when the timescales of both 
processes are systematically varied. 

More precisely, we consider a model of network assembly and evolution given 
by the following steps 

1. Start with a small random network. 

2. Add a new node. The new node forms k connections to randomly selected 
'old' nodes. 




(1) 
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3. Let the network evolve for optimal shortest pathlength. In more detail, 
randomly select a node n and rewire one of its links to a randomly chosen 
new neighbor. If the average shortest pathlength of this node, i.e. d„ = 
1/(A^ — 1) J2i^n '^("'' reduced this way, the rewiring is accepted. If not, 
we proceed with the original network configuration. The optimization step 
3. is repeated for T times, if T is not an integer or T < 1 the fractional part 
of T is interpreted as a probability that step 3. takes place an additional 
time. By normalizing the timescale of the assembly process to one unit of 
time, one can interpret the parameter T as the ratio of the timescales of 
optimization and assembly. 

4. Proceed with 2. until the final network size has been reached. 

Later, instead of step 3. we also consider a variant of the optimization, where 
the search and rewiring is local. In this case we restrict new target nodes of 
the rewired link from n to the neighborhood of n, i.e. nodes up to range r = 2 
away from n. An illustration of local rewiring schemes is given in figure [TJ The 
motivation for considering this local step is twofold. First, one may associate 
it with limited information of agents that try to improve their position in the 
network, but only have local knowledge, i.e. knowledge about their neighbors. 
Second, one may also think about it as resulting from resource limitations when 
the nodes are embedded in space, such that agents can only form connections 
to nodes already close by. 

Before proceeding, let us first briefly revisit the static problem, i.e. T ^ 1, 
treated, eg., in [141 [7]. Shortest pathlengths in a network with given number of 

6 



Figure 1: Illustration of the local rewiring. Node A is the target node and 
node C , which is r = 2 steps away from A is chosen as the new destination 
node. Extending the requirement for strictly local rewiring at range r — 2^ new 
destination nodes can be at distance a distance r > 2 from A. For instance, 
node E would be a possibly choice for a destination node for r = 3. 

links are realized in star-like networks. Depending on the number of links, a core 
with a dominant hub node, which links to all other nodes will form. Nodes other 
than the hub node have low degrees and, apart from the connection to the hub 
node, they have only a few connections to other such periphery nodes. Thus, 
pathlength-optimal networks are very heterogeneous networks, distinguished by 
the presence of a super hub node. It is worthwhile to note that the situation we 
treat here differs from |14( |7] in that nodes optimize their respective shortest 
pathlengths competitively, attempting to maximize individual fitnesses without 
consideration for the global good. In principle, in particular when link costs 
are explicitely considered, under a competitive scheme globally optimal config- 
urations can become unstable [lOj . However, since the optimizing nodes only 
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rewire their connections and do not eliminate links, numerical simulations indi- 
cate that the pathlength-optimal star-configuration is the only globally stable 
attractor of the dynamics. 

As for the second limiting case T <C 1, random assembly, it is known that 
growing networks where new nodes form connections to old nodes with equal 
probability leads to networks with exponentially decaying degree distributions 
[4]. Thus, importantly, the attachment process does not lead to hub formation, 
SF degree distributions or SW behaviour. 

3 Global Optimization and Growth 

In the following we numerically construct large networks of different sizes for sys- 
tematically varied values of T . By displaying the degree distributions for some 
prototypical situations figure [5] summarizes some of the main results. Depend- 
ing on the ratio between assembly and optimization timescales, four parameter 
regimes can be distinguished. In the first two regimes the degree distribution 
can be described by 

P{k) (X k'°'F{k/kN), (2) 

where F(x) is a finite-size scaling function with F{x) w const, for a; ^ 1 and 
F{x) oc exp(— .t) for a; 3> 1. The quantity k^ gives a typical system size 
dependent cut-off degree at which the power law behaviour breaks down due 
to the finite system size N and an exponential decay sets in. For SF networks 
one has fcjv oc , with S > such that the power law regime in the degree 
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Figure 2: Degree distributions of networks with (k) = 2 evolved for (a) T = 10, 
(b) T = 70, (c) T = 350 and (d) T = 1000 for network sizes N = 625, N = 1250, 
2500, 5000 and 10000. The Unes indicate power laws with exponent a = 2. Panel 
(b) additionally compares to a network of size N = 10000 constructed for T = 
(no optimization). Averages are over at least 10 networks and data have been 
binned exponentially. 
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distribution grows with increasing system size [3]. 

The first regime in figure [3] corresponds to the case of small optimization 
times T, cf. panel (a) for T = 10. One finds that the degree distributions are 
well described by a power law with exponent a = 2 for small degrees. However, 
the probabilities of larger degrees decay exponentially. The coincidence of the 
curves for system sizes of different orders of magnitude in panel (a) demonstrates 
that the cut-off length fc^r is independent of the system size, such that the 
resulting degree distributions have essentially exponentially decaying tails. 

With increasing T, typical cut-offs for exponential behaviour are shifted to- 
wards larger degrees, till eventually power law behaviour with exponent a = 2 
governs all scales (and kpf grows algebraically with the system size) , cf. panel (b) 
for T = 70. For larger T, low degree nodes start to lose connections dispropor- 
tionately, while a fat tail of hub nodes extends to larger and larger degrees [panel 
(c) for T — 350]. Finally, 'super-hub' nodes appear, that are well-separated from 
the hub nodes in the power law tail, cf. panel (d) for T = 1000. 

We proceed with an analysis of the networks in the different regimes. To 
characterize the networks, we analyse them in terms of the maximum degree 
A:max = maxi ki , average clustering coefficient |22j , degree correlations as mea- 
sured by the assortativeness a defined in [18], and the average shortest path- 
length d. Figure [3] gives the dependence of these key network statistics on T. 
Notably, increasing the timescales of the optimization process T promotes hub 
formation [cf. Fig. while also leading to more and more cliquish [cf. Fig. 
[3^] and disassortative [cf. Fig. [St] network arrangements. Not surprisingly 
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Figure 3: Dependence of network characteristics on the ratio of optimization 
and assembly time scales T: (a) clustering coefficient C, (b) max. degree fcmax, 
(c) assortativeness a, and (d) average shortest pathlength d. The data have 
been collected for networks of size N = 5000 and represent averages over at 
least 100 independent runs per value of T. 
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Figure 4: Dependence of the normalized clustering coefficient on degree for 
N = 10000 and various values of T. Clustering coefficients are normalized by 
their averages C. Note the change at around T — 70, when clustering starts 
to decay with degree. Power-law behaviour C{k) ^ k^^ (straight lines indicate 
best fits) for large k points to a hierarchical organization: /? — 0.56 for T = 200, 
0.26 for T = 100, 0.07 for T = 70, -0.16 for T = 50 and -0.9 for T = 20. 



also, larger shares of optimization time compared to assembly time cause the 
networks to become smaller and smaller [cf. Fig. [3Ji]. 

In many real- world systems, the cliquishness appears independent of system 
size and the arrangement of cliques is hierarchical [20]. We have conducted a 
scaling analysis of the dependence of the clustering coefficient on the degree and 
on the system size. The first is well described by a power law dependence 

C{k) - fc-^ (3) 
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for large k, with exponents /3 depending on T. Generally, for small T positive 
f3 is found, but the exponents decrease quickly to values around (3 = 0.5 when 
T is increased, thus indicating a hierarchical organization of cliques for large 
T > 100, see figure |4l The parameters around T — 70 for which networks 
in which the power law dependence P{k) oc fc^" with a « 2 extends over 
all scales of the degree distribution [Fig. [2b] have been generated, represent 
the boundary case between growth and decay in the C(fc)-dependence. This 
indicates two different mechanisms to achieve small networks for small and large 
T. Essentially, for small T no tail of hubs is formed. The network evolution 
generates a strongly cliquish core of nodes with larger than average degree. 
For large optimization times, the optimization procedure weeds out connections 
between hubs and hub competition leads to a less hierarchical organization of 
the hubs. This organization can be further classified by correlations between 
degrees of adjacent nodes. 

Given the set of neighbors N'{i) of a node i, let us define the average neighbor 
degree of i via 



and distinguish classes of nodes by degree. Following [19] we then introduce a 
nearest neighbor degree function k^nik), that gives the average over all nodes 
with degree k. Simulation data for this function reveal that A;nn(fc) generally 
follows a power law 




(4) 
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Figure 5: Dependence of the average nearest neighbor degree on the degree for 
N = 10000 and various values of T. Average nearest neighbour degrees are 
normalized by their average K. Note the change in the dependence at T = 70. 
The straight lines indicate power laws with exponents 7 = 0.35 for T = 200, 
0.15 for T = 100, 0.05 for T = 70, -0.09 for T = 50 and -0.4 for T = 20. 
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for large k. As already expected from the dependence of the aggregate assor- 
tativeness on T (cf. figure , a strong dependence of nearest neighbor corre- 
lations on T is notable. For small T one has 7 < and observes a transition 
towards 7 > as T is increased, cf. figure [5j The range of parameters around 
T w 70 again marks the boundary at which 7 changes its sign. Hence, for 
T < 70 networks form a hierarchy of large degree nodes that are preferentially 
linked to each other. Forming links to hub nodes of slightly smaller degrees the 
hub node with the largest degree is located in the centre. Hub nodes of this 
next level form links to nodes of slightly smaller degree while low degree nodes 
are mostly linked to each other and are located at the periphery of the network. 
This contrasts with typical network structures generated for large T, when the 
optimization process weeds out redundant connections between the hub nodes. 
In this case, central hubs preferentially link to low degree nodes, entailing a 
strongly disassortative arrangement. 

Similar to the preferential attachment model an analysis of the scaling 
with system size manifests that the clustering eventually decays to zero in the 
limit of very large systems (data not shown) . 

In sum, the interaction of assembly and global optimization processes at 
various timescales can generate very skewed network topologies. A transition 
timescale ratio T « 70 exists at which degree distributions follow a power law 
with exponent a — 2. Similar to the model of Ref. [IT, explaining power law 
degree distributions by the above mechanism would require the fine-tuning of 
the timescale ratio to T w 70. Below, we will extend the model towards local 
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rewiring procedures. As we will see, in this case a broad range of timescale ratios 
for which degree distribitions fohow power laws exists such that no fine-tuning 
of parameters to obtain power-laws is required. 

4 Local Optimization and Growth 

The above finding motivates us to also consider a different optimization process, 
where rewirings are restricted to local connections (cf. figure [T] above) . Note, 
that in such a local rewiring process the number of triangles can only increase 
(if the rewired connection was not part of a triangle) or stay the same (if the 
rewired connection was part of a triangle). Nevertheless, since the presence of 
communities in sparse networks does not promote short pathlengths, clustering 
coefficients stay well below their maximum values even for relatively large T . 

It is also worthwhile pointing out that the rewiring process without selec- 
tion does not lead to very skewed or SF degree distributions. The reason is 
that even though nodes attract new connections in proportion to their degree, 
connections are also chosen for rewiring in proportion to a node's degree. Both 
effects compensate such that the resulting degree distributions remain narrow. 
A further immediate observation is about characteristic timescales of network 
optimization subject to local or global rewiring. Essentially because in local 
rewiring new destination nodes are suggested in proportion to the destination 
nodes' degree the likelihood of accepting a rewiring suggestion is enhanced. For 
this reason the local optimization is much more effective than the global process 
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Figure 6: Degree distributions for networks of size N — 50000 constructed 
by the local optimization scheme with range r — 2 for (a) T ~ 0.7, 1.0 and 
1.4 and (b) T — 3.0, 4.0 and 5.0. In panel (a) the lines indicate power laws 
with exponents a = 4.35, 3.69 and 3.3, in (b) they indicate power laws with 
a — 2.8 for small degrees and a = 2.0 for the tail of large degrees. At around 
T = 5.0 'super-hubs' start to emerge. Averages over 30 networks, distributions 
are binned logarithmically. 

and characteristic timescales of the local optimization are much shorter than for 
the global process. 

With the data for degree distributions of networks assembled subject to local 
rewiring in figure|6]we proceed with an analysis of the local optimization scheme 
for range r — 2. Apart from different characteristic timescales, results for the 
degree distributions are analogous to the case of globally optimized networks. 
Very short optimization timescales do not allow for the formation of networks 
with very skewed degree distributions. However, from around T — 0.7 onwards, 
degree distributions start to exhibit power law tails. Increasing the optimization 
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timescale, the exponent characterizing the power law tails declines. However, as 
panel (a) of figure [5] demonstrates the distributions still clearly exhibit power- 
laws over around two orders of magnitude. For example, we find power laws with 
exponents a = 4.35, 3.69 and 3.3 for T = 0.7, 1.0 and 1.4. At around T 1.5 a 
tail of hub nodes that deviates from the power law behaviour for nodes of small 
degree develops, cf. panel (b). For T < 5 this tail is well-described by a power 
law with exponent a = 2. At around T « 5 'super-hubs' emerge and gradually 
attract more and more connections when further increasing T . 

A diagram of further average network characteristics such as clustering co- 
efficients, maximum degrees, assortativeness or pathlengths also displays very 
similar characteristics to the one for the global optimization scheme displayed 
in figure [3l but we don't discuss them in detail here. 

It is of interest to note that, even though power law tails emerge for a 
broad range of timescale ratios, local rewiring only allows for the assembly of 
networks with power-law degree distributions with exponents a > 3. However, 
most real-world networks in biological and other applications have been found 
to have exponents in the range 2 < a < 3. Hence, optimization and strictly 
local rewiring with range r = 2 cannot provide explanation for the structure 
of real-world networks. However, as we will see below, relaxing the constraint 
of strictly local rewiring, a much larger configuration space of possible network 
topologies can be explored. 
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5 Between local and global 

Above, we considered a local rewiring scheme where rewirings were always di- 
rected towards nodes at distance r = 2 from the target node. One can extend 
this concept by gradually extending the set of new destination nodes to nodes 
not farther away than an arbitrary range r > 2 P] (and refer to figure [T] for an 
illustration). Since r — > cx) recovers the case of global optimization, tuning r 
allows to interpolate between the local and global optimization schemes. Let 
us assume a fixed timescale ratio T between network assembly and optimiza- 
tion. From the analysis of the limiting cases of global and local rewiring in the 
previous sections, one can understand the implications of a rewiring parameter 
r > 2 on the evolved network topologies. Increasing r to values that are close 
to the characteristic pathlengths of the network reduces the "efficiency" of the 
optimization and dominant hub nodes are less likely to appear. Because of this 
a scenario for a choice of ri , Ti , corresponds to a scenario r2 < ri with T2 < Ti . 
Hence, one expects that an increase in r allows for the assembly of networks 
with less skewed degree distributions. Trivially, relaxing the constraint for local 
rewiring will also reduce cliquishness. 

Interestingly, similar to what we observed for T < 1.5 for r = 2 above, for 
different values of r networks with different steepness of the decay of the degree 
distribution can be constructed. However, unlike for r = 2, for r > 2 degree 
exponents in the range 2 < a < 3 can be found. This is demonstrated by 
the simulation data displayed in figure [7^, in which we compare some example 
degree distributions for networks generated with r = 3 and different T. A 
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Figure 7: (a) Degree distributions of networks generated for N = 50000, r = 3 
and T = 0.7, 1.4 and 10.0. The lines indicate power laws with exponents 
a = 3.8, 3.0 and 2.28, respectively, (b) Dependence of the exponents a on 
the optimization timescale T for different ranges r = 2 (squares), 2.5 (spheres), 
r = 3 (triangles), and r = 4 (diamonds). The values have been determined 
from best fits to the logarithmically binned tails of the degree distributions 
constructed from at least 10 networks of size N = 50.000. For r = 2, 2.5, 3 and 
4 at approximately T = 1.5, 5, 10 and 20 the power laws describing small and 
large degrees start to diverge, cf. text. 



20 



more comprehensive analysis of degree distributions that can be obtained for 
various choices of r and T is given in figure [7|d. Two observations are in order. 
First, increasing r beyond strictly local rewiring for r — 2, degree distributions 
with exponents a in the range 2.1 < a < 4.5 can be obtained. The larger r 
the smaller the exponent characterizing the degree distribution. Second, the 
power law regime is again limited to a range between very low and large values 
of T, outside of which either the assembly process dominates or when hub 
formation sets in. It is difficult to give strict bounds for the range of T values 
for which power-law behavior is found, but the interval covers about one order 
of magnitude for rewiring between global and strictly local rewiring (e.g. from 
about T = 0.5 to T = 20 for r = 4). Clearly, no fine tuning of parameters is 
required to find scale-free networks. 

Another important point is, that also for 2 < r < 3 strongly cliquish net- 
works are formed. This is illustrated in figure [51 which gives some examples of 
the change in cliquishness with growing system sizes for r = 3 and different T. 
It is important to note that unlike the case of the dependence of clustering on 
the system size for global optimization, for every T > clustering coefficients 
converge towards constant non-trivial values for large systems. 

6 Summary and Conclusions 

In summary, in this paper we have analyzed a class of networks that emerge from 
the interplay of a network assembly process and an optimization process. A sys- 
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Figure 8: Scaling analysis of the average clustering coefBcient C with system 
size for local rewiring with r = 3. Note, that C converges to a constant value 
in the limit of large system sizes. 
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tematic investigation of the parameter space reveals that the ratio of timescales 
T between both processes is an important determinant of network structure. For 
low T, network assembly dominates and the constructed networks have narrow 
degree distributions. In the opposite case, for large T, network optimization 
dominates, leading to the formation of networks with bimodal degree distribu- 
tions. These networks are characterized by a periphery of low degree nodes and 
a core of a few "super-hub" nodes. For T ^ oo they evolve into star networks. 

Interestingly, we have found a parameter range "in between", in which the 
structural influences of network assembly and network optimization are bal- 
anced. In this parameter regime we find network structures exhibiting power 
law degree distributions with exponent 2 which are further classified by a hier- 
archical organization, non-trivial degree mixing and very short pathlengths. 

Additional to its characteristic timescale T, the network optimization process 
can be classified by a second parameter, the locality r of the rewiring mecha- 
nism. The parameter can be understood as a measure for the information about 
the system that is available to individual nodes. Varying r we have explored 
situations ranging from strictly local rewiring (each node has only information 
about and access to its immediate neighbors) to global rewiring (every node has 
global information and access). Our analysis points out that this mechanism 
can generate a very rich variety of SF complex networks with various power law 
exponents in the range 2.1 < a < 4.5, all of which also exhibit non-trivial clus- 
tering and interesting patterns of degree mixing. Importantly, the mechanism 
leads to the emergence of power laws for a broad parameter range, such that 
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no fine-tuning of parameters is required to explain SF befiaviour from network 
optimization. 
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